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A Method and System for Super Resolution 
FIELD OF THE INVENTION 
The present invention relates to super resolution enhancement of 

sampled data. 

BACKGROUND OF THE INVENTION 

Super-resolution refers to the enhancement of resolutions for 
images of scenery or objects acquired by an image capture device, such as a 
scanner or a CCD video camera, beyond the resolution inherent in the capture 
device, and beyond other limitations of the capmre device. Super-resolution is 
used primarily to enhance the quality of captured images, and to increase the 
information content therein. 

Many super-resolution techniques used with imaging sensors are 
based on micro scanning operations. When using micro-scanning operations, a 
scene or object being captured is sampled multiple times, each dme with a sub 
pixel shift applied. 

Prior art techniques for super-resolution do not achieve exact 
super resolution, but rather they perform over-sampling. Exact super resolution is 
limited by the fill factor; i.e., the ratio between the area of the region sensitive to 
radiation and the pitch. The pitch is the area between centers of adjacent detectors 
in a sensor panel - equivalendy, the pitch is the area between adjacent pixels. For 
example, if the sensitivity to radiation only extends over half the distance 
separating detectors, then the fill factor is OS * 0.5 = 25%. With a 25% fill factor, 
exact super-resolution using prior art methods can be obtained up to a factor of 
two in resolution in each dimension, since the regions of sensitivity to radiation 
are only half the size of the pitch, and can thus be shifted by half-pixels in each 
dimension without overlapping. Additional super-resolution, beyond a factor of 
two, has been achieved by prior art methods at the expense of decreased contrast 

For a fill factor of 80%, which is typical for image sensing 

devices, the limitation on exact super-resolution is V^i — »1.12. The 

/VO.8 

experimental barrier is approximately at a factor of 1,5, with some decrease in 
contrast. 
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Prior art methods for obtaining super resolution from captured 
images are based on estimating signal distortion. These methods perform optimal 
estimation of signal distortion from the captured images, using Bayesian 
techniques and using criteria such as Maximum Entropy. These methods are 
advantageous when image acquisition is made over a large distance (such as 
satellite data acquired from outer space), since atmospheric transmission and 
turbulence have a major impact on limiting the resolution obtained. 

One such method is described in 1. Cheeseman, B. Kanefsky, R. 
Kraft, J. Stutz, R. H[anson, "Super-Resolved Surface Reconstruction from 
Multiple Images," in Maximum Entropy and Bayesian Methods, G. R. Heidbreder 
(ed.), Kluwer, the Netheriands, 1996, pgs. 293 - 308. This method is based on 
inverse graphics theory, and is used for ground modeling from outer space 
observations. An initial ground model is formed by letting each pixel "vote" on 
what the corresponding ground position should be, based upon the extent that the 
corresponding ground position contributes to that pixel. The initial ground model 
is then used to project what an image should be (i.e., predict each pixel value). 
The differences between the predicted pixel values and the observed pixel values 
are used to update the ground model until it cannot be further improved. This 
procedure produces an increase in both spatial resolution and gray-scale 
resolution. 

Another such method is described in A. Zomet and S. Pele*' 
"Applying Super-Resolution to Panoramic Mosaics," tffp Workshop on 
Applications of Computer Vision, Princeton, Oct 1998. This method attains super 
resolution using an iterative method for mosaicing. Given a video sequence 
scanning a static scene, a panoramic image can be constructed whose field of view 
is the entire scene. Each region in the panorama is covered by many overlapping 
image segments, and this can be exploited to enhance its resolution. 

Another such method is described in Pearson, T. J. and Readhead, 
A. C. S. "Image Formation by Self-Calibration in Radio Astronomy," Ann. Rev. 
Astron. Astrophys. 22, 1984. pgs. 97-130. This method uses non linear methods, 
and is based on optimal spectrum estimation. 
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SUMMARY OF THE INVENTION 

The present specification concerns data obtained by sampling a 
contmuous signal, and describes methods and systems for enhancing the resolution 
of the data. For example, a barcode reader samples a barcode when scanning it 
and the present invention can be used to enhance the resolution of the sampled 
data, thereby providing a better reconstruction of the barcode. For another 
example, a CCD camera samples an object or scene being viewed, to produce a 
digital image, and the present invention can be used to enhance the resolution of 
the digital image, thereby providing a better quality image. 

The present invention enhances sampled data by effectively 
decreasing the sampling period. When used with digital images, the present 
invention provides sub-pixel accuracy. An original image quantized into pixel 
area elements can be enhanced using the present invention to a finer granularity 
quantization with sub-pixel area elements. For example, the enhanced image can 
have one-eighth pixel granularity. 

The present invention overcomes the limitation of prior art 
methods to achieve exact resolution improvemem by more than a factor of 1.5. 
Using the present invention, any desired resolution improvement may be obtained 
by attaching an appropriately designed mask to a sensor plane of a capture device. 

There is provided in accordance with a preferred embodiment of 
the present invention a method for enhancing the resolution of an image sensing 
device, including the steps of attaching a mask to a panel of detectors in an ima-e 
sensing device, generating multiple fields of view, the multiple fields of view 
being related to one another by sub-pixel shifts, acquiring multiple images with 
the image sensing device from the multiple fields of view, and combining the 
multiple images into an enhanced image of higher pixel resolution than the pixel 
resolutions of the multiple images. 

There is further provided in accordance with a preferred 
embodiment of the present invention a system for enhancing the resolution of an 
image sensing device, including an image sensing device comprising a panel of 
detectors, a mask attached to said panel of detectors, a motion generator 
generating multiple fields of view, the multiple fields of view being related to one 
another by sub-pixel shifts, image acquisition circuitry housed within said image 
sensing device acquiring multiple images from the multiple fields of view, and a 
combiner combining the multiple images into an enhanced image of higher pixel 
resolution than the pixel resolutions of the multiple images. 
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There is still further provided in accordance with a preferred 
embodiment of the present invention a method for enhancing the resolution of an 
image sensing device, including the steps of creating replicas of fields of view 
using an optical element attached to an image sensing device, acquiring multiple 
images with the sensing device from the replicas of fields of view, and combining 
the multiple images into an enhanced image of higher pixel resolution than the 
pixel resolutions of the multiple images. 

There is additionally provided in accordance with a preferred 
embodiment of the present invention a system for enhancing the resolution of an 
image sensing device, including an image sensing device, an optical element 
attached to the image sensing device, the optical element being such as to create 
replicas of fields of view, image acquisition circuitry housed within the image 
sensing device acquiring multiple images from the replicas of fields of view, and a 
combiner combining the multiple images into an enhanced image of higher pixel 
resolution than the pixel resolutions of the multiple images. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will be more fuUy understood and 
appreciated from the following detailed description, taken in conjunction with the 
drawings in which: 

Figure 1 is an illustration of a one-dimensional cross-section of a 
prior art image sensing device, such as a CCD camera or a scanner; 

Figure 2 is a prior art illustration of a pixel sensitivity 
(apodization) function, for modeling averaged light intensity arriving at a detector; 

Figures 3A - 3C are illustrations of the results of computer 
simulations of a preferred embodiment of the present invention, as applied to CCD 
cameras; 

Figures 4A - 4D are further illustrations of the results of 
computer simulations of a preferred embodiment of the present invention, as 
applied to CCD cameras; 

Figures 5A - 5C are illustrations of the results of computer 
simulations of a preferred embodiment of the present invention, as applied to 
barcode readers; 

Figure 6 is an illustration of three samplings of an object by a 
sensing device, separated by sub-pixel shifts, in accordance with a preferred 
embodiment of the present invention; 

Figure 7 is a simplified Ulustration of a mask attached to a sensor 
panel of array detectors in an image sensing device, in accordance with a preferred 
embodiment of the present invention; 

Figure 8 illustrates the shape of a mask in accordance with a 
preferred embodiment of the present invention; 

Figures 9A - 9J illustrate the use of a mask in obtaining super- 
resolution, in accordance with a preferred embodiment of the present invention; 

Figure 10 illustrates an enhanced image obtained by applying 
super-resolution to the low resolution CCD captured image illustrated in Figure 
3B, without the use of a mask; and 

Figure 11 is a simplified illustration of an alternating 
("checkerboard") mask attached to a sensor panel of array detectors in an image 
sensing device, in accordance with a preferred embodiment of the present 
invention. 
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DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT 



The present invention concerns digital images captured by digital 
image acquisition devices, such as digital cameras and scanners, and provides a 
method and system for obtaining super-resolution imagery. Super-resolution 
refers to the ability to obtain a resolution of a digital image that is greater than that 
native resolution of the image acquisition device. The present invention uses 
multiple low resolution captures of a picture to obtain a high resolution digital 
image. For example, if 64 images of an object are captured by a CCD camera 
having a spatial resolution of 32 x 32 pixels, the present invention can be used to 
integrate the captured data and provide a digital image of the object at a spatial 
resolution of 256 x 256 pixels. The present invention makes efficient use of 
captured image data, in that there is a direct one-to-one relationship between the 
number of images captured and the ratio of resolution enhancement That is, if iS: 
images are captured then the present invention enhances resolution of the capture 
device by a factor of iC. 

Additionally, the present invention applies to acquisition devices, 
regardless of their dynamic color range. In the example described above, the CCD 
camera can have as low as 1 bit per pixel color depth, or as high as 8 bits per pixel 
and higher, and the present invention is applicable. 

As used in the present specification in reference to digital images, 
the terms "low resolution" and "high resolution" are intended to be relative 
terms, used to indicate an advantage of the present invention in providing digital 
images of an input object at a higher resolution than those produced by an image 
acquisition device. 

Reference is now made to Figure 1, which illustrates a one- 
dimensional cross-section of a prior art image sensing device, such as a CCD 
camera or a scanner. An image sensing device 100 consists of an array of 
detectors 110 in a sensing panel (not shown), each of which measures a light 
intensity of a scene 120 at a specific pixel location. Light rays 130 emanating 
from scene 120 are diffracted by a lens 140, and arrive at detectors 110. 

The array of detectors 110 in the sensing panel is two- 
dimensional, and the one-dimensional array of detectors Ulustrated in Figure 1 is 
but a single cross-section of the two-dimensional array. Similarly, Figure 1 
illustrates a single cross-section of scene 120. The direction of the one- 
dimensional array of detectors is denoted by a variable x, and the light intensity of 
the scene is correspondingly denoted by u(x). The variable x is a continuous 
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vanable, and scene 120 corresponds to a continuous signal. The array of detectors 
110 samples scene 120, and produces a discrete digital image, comprising pixel 
0 at.ons and p.el color intensities. The spacings between detectors corresponds 
to the pixel size for the sampled digital image, and is referred to as the "pitch " 

Reference is now made to Figure 2, which Ulustrates a one- 
dimensional cross-section of a prior art pixel sensitivity (apodization) function, for 
modeling averaged light intensity arriving at a detector. Th. measured licht 
•ntensity at each detector 110 (Figure 1) is not a measure of the light intensity al a 
single pomt location of scene 120. Instead, it is an average intensity, averaged by 
a pixel sensitivity function gM, or apodization. The pbcel sensitivity function is 
Illustrated as a one-dimensional function g(x) in Figure 2, but g(x) is only a single 
cross-section of a two-dimensional pixel sensitivity function g(x, y). 

The support of the two-dimensional pixel sensitivity function 
corresponds to the region that is sensitive to radiation. This region typically 
extends across an area that is contained within the area between neighboring pixel 
centers. The ratio between the area of the two-dimensional support of the pixel 
sensitivity function and the area between pixel centers containing it is referred to 
as the "fill factor." 

It is the averaging with g(x) that gives rise to complications in 
obiammg exact super-resolution in prior art methods. 

Reference is now made to Figures 3A - 3C. which illustrate the 
results of computer simulations of a preferred embodiment of the present 
invention, as applied to CCD cameras. Figure 3A shows an input picture with 
spatial resolution of 256 x 256 pixels. Figure SB shows an image of the input 
picture, captured by a CCD camera having a spatial resolution of 32 x 32 pixels 
and 8 bits per pixel color depth. Figure 3C shows the reconstructed image' 
obtained by the present invention by using 64 pictures like the one illustrated in 
Figure 3B. It can be seen that substantially all of the spatial details contained 
withm the mput picture are reconstructed in the image of Figure 3C. 

Reference is now made to Figures 4A - 4D, which further 
Illustrate the results of computer simulations of a preferred embodiment of the 
present mvention, as applied to CCD cameras. Figure 4A shows a low resolution 
image captured from the input picture illustrated in Figure 3A, using a CCD 
camera havmg a spatial resolution of 32 x 32 pixels and a color depth of 1 bit per 
pixel. It can be seen that the image shown in Figure 4A is a black and white 
image. Figure 4B shows the reconstructed image obtained by the present 
invention by using 64 pictures like the one illustrated in Figure 4A. It can be seen 
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that although Figure 4B is not identical to Figure 3A, it nevertheless is a much 
better approximation than is Figure 4A, both in terms of spatial resolution and in 
terms of dynamic color range. 

Similarly, Figure 4C shows a low resolution image captured from 
the input picture illustrated in Figure 3A, using a CCD camera having a spatial 
resolution of 32 x 32 pixels and a color depth of 2 bits per pixel. It can be seen 
that the image shown in Figure 4C is comprised of four colors. Figure 4D shows 
the reconstructed image obtained by the present invention, by using 64 pictures 
like the one illustrated in Figure 4C. It can be seen that Figure 4D is almost 
identical to Figure 3 A. 

When the present invention was applied to a CCD camera having 
a spatial resolution of 32 x 32 pixels and a color depth of 4 bits per puxel (not 
shown), the reconstructed image (not shown) was indistinguishable from the 
original. 

Reference is now made to Figures 5A - 5C, which illustrates the 
results of computer simulations of a preferred embodiment of the present 
invention, as applied to barcode readers. Shown in Figure 5A is an original 
barcode. Shown in Figure 5B is the image captured by a bar code reader. As a 
result of the limited resolving capability of the barcode reader, the captured image 
shown in Figure 5B is substantially different from the barcode shown in Figure 
5A. Shown in Figure 5C is a reconstructed image of the barcode using the present 
invention, based on eight vertical replications. A comparison between Figure 5A 
and Figure 5C indicates that the present invention successfully overcomes the 
limitations of conventional barcode readers in resolving scanned barcodes. 

The present specification describes the process for acquiring 
multiple low resolution images from a single picture, and the way to combine the 
multiple low resolution image data to generate a high resolution image. 

Conventions an d Notatinn 

In the ensuing description it is assumed that a sensing device 
samples an object being acquired in lines, such as horizontal lines, using pixels 
per line. The present invention is preferably applied separately in two dimensions, 
such as in a horizontal dimension and a vertical dimension, to increase the pixel 
resolutions in each of the dimensions. Since the applications of the present 
invention in each of the dimensions are similar, for the sake of clarity and 
defmiteness the present specification describes a preferred embodiment of the 
present invention in a single dimension, such as a horizontal dimension. In 
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particular, rather than use two coordinates (x, y) in the discussion below, a single x 
coordinate is used, and rather than work with double integrals and double suras, 
single integrals and single suras are used. Similarly, two-dimensional doubly 
periodic masking of a two-dimensional array of detectors in a sensing panel is 
described hereinbelow using a one-dimensional periodic masking function m(x). 

When the sensing device samples a line, the detected energy of 
the n-th pixel is given by 

so 

u[n] = ju(x + nAx)g(x)dx , n = 0, 1, N-2 , (1) 

where u(x) denotes the captured object along a specific line, /ix denotes the pixel 
width and g(x) denotes the pixel sensitivity (i.e. apodization). 

Multiple Image Acquisitinn 

In a preferred embodiment of the present invention, the sensing 
device samples a line K times, each time with a shift in the captured object relative 
to the capture device by an amount of Ax / K. Letting ukfn] denote the detected 
energy of the n-th pixel in the k-th sampling. Equation 1 generalizes to 

«* [«] = ju(x + n^ + k —)g(x)dx , 

n = 0,l, N-1; k = 0,I, K-1 . (2) 
Together, the various samples ukfnj provide KN samples given by 

" Ax 

y[n] = ju(x + n Y)g(x)dx , n ^ 0. 1, KN-1 . (3) 

"CO 

In an preferred embodiment of the present invention, the shifts in 
successive image acquisitions hy Ax / K are implemented by using vibrations of 
the sensing device. For example, the sensing device can be placed on a vibrating 
platform. The parameters of the vibrations are estimated by vibration sensors or 
by use of appropriate algorithms. The vibration parameters are used to 
synchronize the sampling of the object so that the sampling occurs at times when 
the vibrated sensing device is located at appropriate sub-pixel shifts. 

In an alternate embodiment of the present invention, an assembly 
of one or more rotated mirrors can be used to shift successive fields of view of a 
sensing device by sub-pixel shifts. The rotated mirrors reflect the scene to the 
sensing device. 

In yet another embodiment, the shifts in successive image 
acquisitions by ^ / ^: can be implemented using an optical element instead of the 
imaging lens of the sensing device. The optical element is attached to the aperture 

9 
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Of the sensing device, and serves like a grating to create replicated views of an 
object. Typically gratings result in replicas since the Fourier transform of a 
grating is an impulse train of delta functions. This embodiment using an optical 
element to create replicated views is particularly weU suited for objects that are 
sufficiently small so that the replicas do not overlap. 

The abovementioned optical element can be designed, for 
example, by a multi-facet lens. It can also be designed by an algorithm such as 
the one described in Z. Zalevsky. D. Mendlovic and A. W. Lohmann, Gerchberg- 
Saxton algorithm applied in the fraaional Fourier or the Fresnel domain. Optical 
Letters 21, 1996, pages 842 - 844, the contents of which are hereby incorporated 
by reference. 

It should be apparent to those skUled in the art that it is not 
necessary that the multiple acquired images be separated by identical sub-pixel 
shifts. The present invention applies to multiple acquired images separated by 
sub-pixel shifts of arbitrary sizes. Moreover, successive shifts between acquired 
images do not have to be sub-pixel. The shifts can be larger than one pixel shifts, 
as long as the multiple acquired images are aU non-integral pixel shifts from one 
another. Specifically, the relative shifts of the multiple acquired images with 
reference to a fixed origin can be set to values r^Jx, r/^, .... rj^./Jx, as long as 
the differences r/- ry are non-integral for any distinct indices / and The case of 
equal sub-pixel shifts described above corresponds tork=:k/K. 

Reference is now made to Figure 6, which illustrates three 
samplings of an object by a sensing device, separated by sub-pixel shifts, in 
accordance widi a preferred embodiment of the present invention. The object is 
indicated by a curve u(x), denoted by reference numeral 610. The curve u(x) 
denotes the color of the object along a line, as a function of position, x. Also 
indicated in Figure 6 is a curve g(x) denoting the sensitivity of a pixel, denoted by 
reference numeral 620. The curve g(x) is a characteristic of the sensing device, 
and indicates how color values are localized by averaging. 

Each row in Figure 6 indicates a sampling. The first row samples 
the object u(x) at positions indicated by dashed vertical lines 630. The six pixel 
values are denoted by uo[0], uQflJ, ... uQfSJ. The second row samples the object 
u(x) at shifted positions indicated by dashed vertical lines 640. The six pixel 
values are denoted by ujfOJ, ujflj, ... ujfSJ. The third row samples the object 
u(x) at shifted positions indicated by dashed vertical lines 650. The six pixel 
values are denoted by U2f0j, U2[l], ... U2[5J. 
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The samplings indicated by the second and third rows is often 
referred to as "sub-pixel" sampling, since these samplings are centered at 
fractional pixel locations. Specifically, in Figure 6, the samples in the second row 
are centered at the one-third pLxel locations, and the samples in the third row are 
centered at the two-thirds pixel locations. The value of K for the geometry 
illustrated in Figure. 6 is K = 3. The three sets of samples together form the 
eighteen samples yfOJ = uofOJ. yflj = ujfOJ, y[2] = U2[0], y[3] = uo[l], 
y[17] = U2[5], form the full set of samples at all of the pixel thirds locations. 
These eighteen samples y[nj are used to reconstruct the values u(0), «(— ), 

"(-T-). «(— r— ) of the object u(x). 

The present invention uses the samples y[nj to approximately 
Ax 

reconstruct the values of «(«—). Detenriiiiing these reconstructed values 

K 

achieves super-resolution with a resolution enhancement factor of K, since the 
sensing device has a resolution of N pixels per line, and the reconstructed signal 
has KN pixels per line. 

In order to describe the reconstruction of the values u{n—) from 

K 

the samples y[n], a classical result from signal processing is used, which relates 
the discrete Fourier transform of a sampled analog signal to the continuous 
Fourier transform of the analog signal. 

A Bit of Hist ory; Frequency Representation of Samnling 

In the ensuing description, s(t) is used to denote a continuous time 
signal, and S(jn) is used to denote its continuous-time Fourier transform, defined 
by: 

SUa)=[j{t)e-'^dt . (4) 

Periodic sampling of 5(t) with a sampling period of T produces a discrete time 
signal 5[n], defined by: 

5[n] = 5(nT) . (5) 
The discrete-time Fourier transform of s[n] is denoted by 5Ce/'";, and is defined 
by: 

5(e>)=|;5Me-'- . (6) 

An age-old result, referred to as the Poisson Summation Formula 
and which is one of the most prominent formulas used in discrete signal 

11 
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processing, provides a relationship between the continuous-time Fourier transform 
S(jr2) of a continuous time signal and the discrete-time Fourier transform S(e}'") of 
the sampled signal. Specifically, the Poisson Summation Formula states that 

|;.(.r) = l|;5(-yM). 

<?— CO i km-n J. 

In fact, more generally the Poisson Summation Formula establishes that 

go ^ 00 ^ f 2jtkx 

Y.^i'*nT) = lY.S(.-i^y"~ , (8) 

of which Equation 7 above is a special case corresponding to f = 0. 

A simple proof of Equation 8, as indicated on page 210 of Rudin, 
W., "Real and Complex Analysis: Second Edition," McGraw-Hill, 1974, is as 
follows. The function /fr; defined by 

/(0= E^C^ + nr) (9) 

is a periodic function with period T. As such, it has a standard Fourier series 
expansion of the form 

f(0=^c,e''^ , (10) 
where the Fourier coefficient is given by 

^jlfiOe ^ dt . (11) 
Upon substituting Equation 9 into Equation 11 it can be seen that 

=^£5(r)e^^rfr = i5(-;^) , (12) 

from which Equation 8 follows when Equation 12 is substituted back into 
Equation 10. 

If the function s(t)e ^ is used in Equation 8, instead of the 
function s([), one obtains the result 

t so + nT)e-^- -~-tsU ^^^W'"^ . (13) 

By setting / = 0 in Equation 13 one arrives at the familiar sampling equation 

Sie"")-\t^U^^') , (14) 

which appears as Equation 3.20 in A. V. Oppenheim and R. W. Schafer, 
"Discrete-Time Signal Processing," Prentice Hall, 1989. Equation 14 indicates 
that the discrete Fourier transform of a sampled analog signal is comprised of 
periodically repeated copies of the continuous Fourier transform of the analog 

12 
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Signal. The copies are shifted by integer multiples of the sampling frequency and 
superimposed. Equation 14 is often expressed in terms of delta functions; namely, 
that the Fourier transform of an impulse train 2^(/~„r) is itself an impuIsJ 
train ft^r(Q>M). 

It is noted that Equation 14 can also be derived directly from 

Equation 7 by using s(t)e''~ instead of s(t). 

For values of oj that are small enough to avoid significant aliasing 
m Equation 14, say axCi,, one can approximate the sums on the right hand sides 
of Equations 13 and 14 by the term with k = 0. When this approximation is made, 
it in turn leads to the approximation 

It is noted that as the sampling period, T. decreases, the approximation in 
Equation 15 becomes more accurate. The increase in accuracy is due to the fact 
that the aliasing, or overlap, between the copies on the right hand sides of 
Equations 13 and 14 decreases, since the shifts ^ between copies increase. 

The present invention uses a version of Equation 15 to obtain 
super-resolution. Specifically, instead of sampling s{nj = s(nT) as in Equation 5. 
the sampling is done through an integral 

yln]= [^s(t + nT)g(t)dt , (16) 
where g(t) represents the sensitivity, or apodization, of a pixel. Multiplying both 
sides of Equation 15 by g(t) and integrating, results in 



<^^,. (17) 



which is the form used in the present invention. 

Referring back to Equation 3, it follows from Equation 17, upon 



replacing r by —.that 

jfC 



0{-J —) 
Ax 



Ax 



< . (18) 



where U(ej'') is the discrete Fourier transform of the samples «(«—), sought to 
be reconstructed; namely, 
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U(en=tu(nf)e-^'- . (19) 
The Fourier transform Y(e.n is known from the samples yfnj in Equation 3, and 
the Fourier transform G(-y~)is determined from knowledge of the sensitivity 

function g(x). It is noted that as K increases, the approximation in Equation 18 
becomes more accurate. 

Regarding the Fourier transform Y(ei'"), the values of y[n] set 
forth in Equations above are only defined for indices n = 0, 1, KN-1. For 
indices n outside of this range, y[nj is preferably defined to be zero. With such a 
convention, the Fourier transform Y(d'") is given by the finite sum 

fOI-X 

Y{e"')^Y.y["]^-"^ ■ (20) 



n«0 



In a preferred embodiment, the present invention uses Equation 
18 to determine the Fourier transform U(ei'-'), and then reconstructs the values 

/C ' "■' ^ " ^' t^fe^") by using an inverse discrete Fourier 

transform. Preferably a Fast Fourier Transform (FFT), as is famUiar to those 
skilled in the art, is used to invert the spectral values U(e}'^. The reconstruction is 
approximate, since Equation 18 is itself only an approximation. Since it is 
necessary to reconstruct A2V values of u(x), it is accordingly necessary to apply 
Equation 18 at KN fi-equencies that are spaced ~r apart one from the next, since 

the Fourier transform U(ef'') is periodic with period 2;r. In a preferred 
embodiment of the present invention. Equation 18 is applied at the BCN frequencies 

^ = ;^. k^O,l,...,KN-l. (21) 

When the Fourier transform U(ei°') is inverted it is possible that 
Ar 

values of "(1—) are non-zero outside of the index range n = 0, 1, KN- 1. As 
it is only the values of u(n^) within the index range n = 0, 1, KN- 1 that are 
of interest, the values outside this index range can be ignored. 

An apparent difficulty arises in implementing Equation 18 if there 

are frequencies a) as given in Equation 21, at which G(-;— ) = 0, since this 

Ax 

would entail division by zero at such frequencies. 

The function G(-J^^ is typically a real-valued ftmction, since the 
pixel sensitivity function g(x) is typically symmetric about x = 0. Since a pixel is 
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typically more sensitive to light in its central area and less sensitive in its outer 
regions, the pixel sensitivity function g(x) is typically "bell-shaped," similar to 
Gaussian functions. In case g(x) happens to be a Gaussian function, its Fourier 
transform has no zeros, and there no divisions by zero are encountered in 
•mplementing Equation 18. However, for other sensitivity functions, such a 
difficulty can arise. 

The present invention avoids such a difficulty by using specially 
constructed masks, as described hereinbeiow. 

Masking 

In order to avoid division by zero in Equation 18. the present 
invention preferably employs an optical device to remove zeros of G(-j^) that 

are less than the frequency /2y. The frequency is the maximal frequency in the 
reconstructed data. 

The present invention preferably attaches a mask in the foim of a 
fine transmission grating with a period of Ax to the sensing device panel. As 
mentioned hereinabove, the mask under discussion is reaUy a two-dimensional 
doubly periodic mask that covers a two-dimensional array of detectors. For the 
sake of clarity and definiteness it is being described as a single dimensional 
periodic mask, with the understanding that the one-dimensional analysis presented 
herein applies to each of the two dimensions of the sensing device panel. 

Reference is now made to Figure 7, which is a simplified 
illustration of a mask attached to a sensor panel of array detectors in an image 
sensing device, in accordance with a preferred embodiment of the present 
invention. A sensor panel 700 in an image sensing device contains a two- 
dimensional array of detectors 710. Detectors 710 are depicted as individual cells 
within sensor panel 700. The array of detectors is periodic, with periodic spacings 
Ax and Ay in the horizontal and vertical dimensions, respectively. The spacings Ax 
and Ay correspond to the pixel spacings in the sampled digital image obtained 
from the image sensing device. A masking pattern 720 in the form of a fine sub- 
pixel transmission grating is used to generate a doubly periodic mask, with 
respective periods Ax and Ay in the horizontal and vertical dimensions. In a 
preferred embodiment of the present invention, the doubly periodic mask is 
attached to sensor panel 700. Each detector 710 is masked by the same masking 
pattern 720. 
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Use of a mask decreases the effective size of each pixel so that the 
Fourier transform G(ja) of its sensitivity function has a wider band, and its zeros 
accordingly move to higher frequencies. The apodized pixel, determined by the 
transmission pattern of the grating, is preferably designed in an iterative manner, 
as described hereinbelow, so that the bandwidth of the Fourier transform CQH) of 
its sensitivity function is wider than the maximal frequency to be reconstructed. 
Use of a transmission grating has the disadvantage of decreasing the energy sensed 
at each pixel, since it blocks out some of the illumination. 

A mask is described mathematically by a spatial function m(x), 
preferably taking values 0 and 1. The effect of a mask is to modify the pixel 
sensitivity function from g(x) to p(x) = g(x)m(x). An objective of choosing a 
suitable mask is to obtain a function p(x) whose Fourier transform does not have 
zeroes at frequencies that are between 0 and /2y, so that when the Fourier 

coK. 

transform ^(--/~) of the modified pixel sensitivity function p(x) is used in 

Equation 18 instead of C(-y ) , there is no division by zero. 

Ax 

A rationale for using a mask to avoid zeroes of G is that typically 
an effect of a mask is to add frequencies that may have not been present 
beforehand. 

To construct a mask as desired, the present invention preferably 
uses a grating in the form of a plurality of optically transparent slits. 

'«W = l-Z''^^<Tr)®^(^-^^i) > (22) 

where M is the total number of slits, xi, ^2, xm are the locations of the slits and 

<&/ is the width of the z-th slit. The function rectQ is given by 

. , \X -l/2<z<l/2, 
rect{z)^\ ' (23) 

[0, otherwise 

and ® denotes convolution. 

Reference is now made to Figure 8, which illustrates the shape of 
a mask in accordance with Equation 22. Such a mask consists of M slits 810, 
centered at positions x;, X2, xm. each having respective widths of &/. 
Preferably,' the value of A/ is a number between zero and K, and the values Sxi are 
each approximately equal to the sub-pixel size /\x f K. As can be seen m Figure 8, 
the values of the mask m(x) are 0 inside the slits and 1 outside the slits. The total 
width of the mask if zIa:; namely, the width of a pixel. The mask is intended to 
affect the shape of g(x) at sub-pixel granularity. In practice, the granularity is 
discretized to ^Sx I K; namely, the sub-pixel shift between successive images 
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acquired by the image sensing device. For example, referring back to Figure 3C, 
the mask used in reconstructing the image shown therein is [0 1 0 1 1 0 0 0], each 
of the eight entries in this array representing a sub-pixel size of one eighth of a 
pixel. Thus the function m(x) for the mask used in Figure 3C is given for 
0 < ,v < Aa: by 

"W^O l^^"^f^''"'|'^^'<f^. (24) 
' Otherwise 

The sensitivity function of each masked pixel is given by 
P(x) = g(x)m(x) , (25) 
and it can be seen that the Fourier transform ofp(x) is given by 
P(jn) = /*(;X2;--c, .j:, ..... Jt.w .<&2 v".«5c.vf ) = 

= G(;Q) - 1; &,G(;X5) ® [e-'""sinc(^n&,)] , (26) 

where sinc(z) denotes the function sinc(2) = ^i2£ , 

z 

Preferably, to determine an optimal mask, a predetermined value 
of M is selected, and a search is made for position values x], xz xm and 
widths 5x1, 3x2, Sxm for which the \P{jCl)\ is bounded away from zero as 
much as possible. Specifically, define 

= gg)l^(/";^i»^2--.^iif ;^i,^2>--..5c,v , (27) 
where iTy denotes the frequency up to which no significant aliasing occurs, as 
defined above. Then optimal values forxj, xz W and Sxi, Sxz &M are 
preferably determined by 

= arg-maxc7(A:^,A:2,...,;cAf ;c&,,<Sc2'-»<Scw) , ,(28) 
where arg-^max denotes the arguments xj, X2, xm and Sxi, Sx2, &M which 
maximize the function cr. 

The search range over which optimal values ofxj, X2, x^ and 
&CJ, 5x2, are sought is such that the total (5c/ + &2 + + of all the 

widths does not exceed the pixel width Ax, Additionally, each individual width 
&ci must not be less than 2a, where Z is the wavelength of the incoming light - 
otherwise, the light will not be affected by the mask. Moreover, construction of a 
mask and attaching it to the detectors may further limit the widths &[ to exceed a 
minimum value of 5c/nm- 

From energy considerations, it is clear that on the one hand, it is 
desirable that each width &/ be as small as possible, since &/ is proportional to 
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the energy blocked by the mask. On the other hand, this must be traded off against 
the condition that |/»(yQ)j be bounded away from zero. 

Specifically, if £; denotes the total light energy for a typical 
viewing target, when viewed without a mask, then the energy detected when a 
mask is present is given by 

^=^'a-^)> (29) 

since the energy blocked by the slits of the mask is proportional to the total width 
of the slits combined. 

Thus for a given minimum energy Emin, required to activate the detectors, the 
widths <&£ are constrained to satisfy 

:S(l-%=-)Ax. (30) 

{•I 

Condition (30), together with the condition that Sx, ^Sx^„, for / = 1, 2, .... M, 
determine the search range for widths &ci, Sxz, dxu over which a maximum of 
a{xi, xz xf^; dxi, 5x2, dx^) is sought. 

When values of xi, xz xm and acj, Sx^ .... Scm are 
determined according to Equation 28, Equation 22 is used to generate an optimal 
mask. 

In and alternative embodiment of the present invention, rather 
than maximize ofxj, X2, xm; &j, Sxz Sxi4), the values of ;cj, xz xm 
and Sxz &,\/are determined by maximizing a weighted function ofxj, xz 
xm; Sxi, Sxz — . Sxm) MSxi, Sxz •••» where w is an appropriate weight 
function. For example, w can put higher weight on smaller values of the widths 
Sxi, so that preference is given to masks having slits with smaller widths. 

Reference is now made to Figures 9A - 9J, which illustrate the 
use of a mask in obtaining super-resolution, in accordance with a preferred 
embodiment of the present invention. Figure 9A illustrates an input object, such 
as a barcode, represented as a function u(x). Figure 9B illustrates the Fourier 
transform of u(x). Figure 9C illustrates a pixel sensitivity function g{x). Figure 
9D illustrates the Fourier transform of g{x). 

Figure 9E illustrates conventional sampling of u{x) as a digital 
signal, without using super resolution enhancement. Figure 9F illustrates the 
Fourier transform of the digital signal of Figure 9E. Figure 9G illustrates 
sampling of u{x) as a digital signal with a resolution eight times higher than the 
sampling illustrated in Figure 9E. Figure 9H illustrates the Fourier transform of 
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the digital signal of Figure 9G. Figure 91 illustrates a mask used in accordance 
with a preferred embodiment of the present invention. 

Figure 9J illustrates the Fourier transform of a masked pixel 
using the mask of Figure 91. By comparing Figure 9D with Figure 9J it can be' 
seen that the spectrum of the masked , pixel does not contain zeros within the 
region displayed. It can further be seen that an effect of the masking is to expand 
the spectrum ofg(x). 

Reference is now made to Figure 10, which illustrates an 
enhanced image obtained by applying super-resolution to the low resolution CCD 
captured image illustrated in Figure 3B, without the use of a mask. As can be seen 
in Figure 10, artifacts in the form of vertically oriented white stripes arise due to 
the presence of zeroes of G(-y ^) , When a mask is applied in accordance with 

a preferred embodiment of the present invention, the zeroes of G(-j^) are 

Ax 

removed, and the result obtained is the image shown in Figure 3C. It may thus be 
appreciated that the use of a mask serves to remove the artifacts indicated in 
Figure 10. 

Improved App ^pximatinn 

As explained hereinabove, the approximation in Equation 18 is 
based on the assumption that q> is small enough to avoid aliasing in Equation 14. 
A more accurate approximation can be made by taking into consideration the 
aliasing between adjacent copies of 5 in the right-hand side of Equation 14. 
However, in order to obtain the increase in accuracy, it is necessary to increase the 
sampling rate. 

It is appreciated by those skilled in the art that the assumption that 
aliasing between copies of 5 in Equation 14. is due to overiap between adjacent 
terms 5(y-) and SiJ-^ — )is equivalent to the assumption that reduction of 

the sampling period from T to T/2 removes aliasing. Moreover, the information at 
frequency co in the discrete Fourier transform of a signal sampled with sampling 
period r corresponds with the inforaiation at both frequencies aj/2 and a}/2 + ;rin 
the discrete Fourier transform of the signal sampled with sampling period T/2. A 
precise formulation of this relationship appears in the above referenced 
Oppenheim and Schafer as Equation 3.76, which describes a frequency-domain 
relationship between the input and the output of a sampling rate compressor 
("downsampling"). 
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These observations above are a basis for an improved 
approximation used in accordance with a preferred embodiment of the present 
invention, as described hereinbelow. Specifically, replace the sampling period, T. 
by TI2 in Equation 16: 

y[«]=£5(/ + n|)^(f)rf/ . (31^ 
Under the assumption that aliasing in Equation 14 is due to overlap between 
adjacent terms SUf) and SU^^),ti^ approximation to S(ej'^ becomes 

Sien.^SU^)^^SU^) , 0<a,<;r . (32) 
for positive values of a>, and 

)~-J(y-) + -.5(^___) ^ -TTKOXO, (33) 

for negative values ot <a. 

Using similar analysis to that embodied in Equations 13 - 18, it 
can be shown that 



2 



0<a><jc , (34) 

for positive values of co, and 



2 



Ax Ax 



-jr<(o<0, (35) 

for negative values of oy. Equations 34 and 35 determine U(ei'') more accurately 

than Equation 18, since they incorporate aliasing between adjacent copies of 5 in 

the sampling representation for S(ei'"). However, they require sub-pixel shifts of 
Ax 

— between successively captured images. This corresponds to a factor of 2A: 
increase in data acquisition, whereas the resolution is only enhanced by a factor of 



An alternative approach to improving the approximarion in 
Equation 18 by taking into consideration the aliasing between adjacent copies of 5 
in the right-hand side of Equation 14, uses an alternating ("checkerboard") mask 
geometry. As mentioned hereinabove, the above discussion is based on use of a 
periodic mask, with a period of /Ix. By using instead a inask with a period of 2/ix, 
one effectively generates two pixel sensitivity functions, gj(x) and g2(x), where 
gl(x) applies to even-numbered detectors and g2(x) applies to odd-numbered 
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detectors. Specifically. g2(x) corresponds to g(x)m(x) and 82(x) corresponds to 
g(x)m(x^Ax), where g(x) is the pixel sensitivity function for the image sensin-^ 
device and mCr; is the overall mask. Unlike the above case where the mask has I 
period of 4.. when the mask has a period of 2^ there are effectively two masks 
operatmg (namely, the mask itself and the mask shifted by Ak), and consequenUy 
the even and odd-nunibered pixels are processed differendy. 

Reference is now made to Figure 11. which is a simplified 
Ulustration of an alternating ("checkerboard") mask attached to a sensor panel of 
array detectors in an image sensing device, in accordance with a preferred 
embodiment of the present invention. A sensor panel 1100 in an image sensing 
device contains a two-dimensional array of detectors 1110. Detectors 1110 are 
depicted as individual cells within sensor panel 1100. The array of detectors is 
periodic, with periodic spacings Ax and Ay in the horizontal and vertical 
dimensions, respectively. The spacings Ax and Ay correspond to the pixel 
spacings in the sampled digital image obtained from the image sensing device. . 

In distinction to Figure 7, Figure 11 illustrates two different 
masking patterns 1120 and 1130 in the form of fine sub-pixel transmission 
gratings that are arranged in an alternating checkerboard pattern to generate a 
doubly periodic mask, with respective periods 2zlr and 2/l>' in the horizontal and 
vertical dimensions. In a preferred embodiment of the present invention, the 
doubly periodic mask is attached to sensor panel 1100. The detectors 1110 are 
alternatively masked by the different masking patterns 1120 and 1130. 

When using such a mask. Equation 2 (which is one-dimensional) 
separates into two equations. 

oo 

u,[n]= luix^nAx + k^)g^{x)dx . 

for even values of n; k^O.l,..., K-1, and (36) 
/«(^ + «Ax + A:^)^,(A:)dr . 

for odd values of n; k^O,l, .... K-1. (37) 
By capturing twice as many images as previously, namely, 2K images, each 
detector moves over a full period of the mask, namely, 2Ax. Consequently, two 
sequences of the form of Equation 3 are acquired, namely, 

y\n\ = \u{x + n —r)g(x)dx , n^O,l,..., KN-1 , and (38) 

* Ax 

[«] = \uix + n Y)8(x)dx , n = 0, I. KN-1 . (39) 
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In turn, using the same analysis embodied in Equations 13 ^ 18, one arrives at the 
two conditions 

y.(^'-)'YSuf)G,(-jf).lsa^)G,(-iii^^ , (40) 

y.i'")''^saf)G,i-jp.isa^)G,i-j^^ . (4i) 

for positive values of co, 0 < o) < ;t, and 

y.(e")^^Saf)a.l-jf).l,S(J^yG,(-i^), (42) 
n(«'«)45(/^)c,(-;£Hi5(;-|2£)C,(-;£±?£) . (43) 
for negative values ofa>,-;r<a><0. These equations can be used to solve for the 
individual copies SUf) and S(J^j^), and then to recover S(ei'") from 
Equations 32 and 33. The final result obtained, expressed in terms of U(ei''), is 



a.(-y^).^.,(-,f)-a,(-;^).^a.(-yf) 



for positive values of oj, 0 < q>< and 



(44) 



4« ^ — Ax 



-=(-;f)^^a,(-,^)-C,(.,^)._^^,(.,^, 



(45) 



for negative values of a, < a, < 0. In Equations 44 and 45. the terms 5vG(-jn) 
denote the difference 

s^(.jn) = G(-jn) - G(-j(n- ^) . (45) 

In distinction to the approach using Equations 34 and 35, whereby 
the improvement in accuracy stems from reducing the sub-pixel sampling shift in 
half, from — to — , the approach using Equations 42 and 43 obtains 

improvement in accuracy by acquiring 2/i: sampled images, each shifted by — . 

Both approaches require collection of sampled data that is 2K as much data as in a 
single image captured by a sensing device. They differ in the types of masks used, 
and in the relative shifts between the multiple images acquired. 

When implemented with full two-dimensional processing, both 
approaches described hereinabove require 4k2- as much data as in a single image 
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captured by a sensing device, in order to enhance image resolution by a factor of 
two in each pixel dimension. 

geometric Super Resolnfion v.. niff...*-^.,. Stin^r R.c^f ».|^^ 

The multiple image acquisition described above and illustrated in 
Figure 6 is geometric in nature; i.e., it is based on viewing the same spatial 
information several times, such that in each view the object is shifted bv a sub- 
pixel amount. It can be implemented by shifting positions of the sensing device 
("temporal multiplexing") or, equivalently. by shifting fields of view ("spatial 
multiplexing") using an optical element that replicates views of the object with 
sub-pixel shifts between the replicas. The resolution of a system limited by 
geometry corresponds to the detection sampling system; i.e., the resolution of 
system sensors. 

In distinction, multiple image acquisition can also be dif&active 
in nature. The resolution of a system limited by diffraction corresponds to the 
finest detail that can pass through the system without being distorted, and is 
propomonal to the size of the aperture in the optical system. Thus the resolution 
of the human visual system is limited by the extent to which the eyes are open 
When an observer squints his eyes, the resolution of a scene being viewed 
decreases, since the size of the opening is decreased. 

For example, resolution enhancement can be implemented by an 
assembly of two moving rotated gratings, a first grating attached to the object 
being captured, and a second grating attached to a sensor of the sensing device 
and moving in a direction opposite to that of the first grating. Such an assembly 
can be used to enhance the resolution of a diffractive system, since it effectively 
increases the size of the aperture in the optical system. The movement of the 
gratmg attached to the object encodes its spatial infonnation by a Doppler-like 
effect, and allows the infonnation to be transmitted through a restricted aperture of 
an imaging lens. The decoding of the information is obtained from the second 
gratmg attached to the sensing device, which moves in a direction opposite to that 
of the first grating. Such an assembly of two rotated gratings is practical for 
microscopic applications. 

However, in many imaging systems the object is distant from the 
observer, and attachment of a grating to the object is impractical. When applied to 
scanning systems such as barcode readers, the present invention preferably 
simulates attachment of a moving grating to the object by illuminating the object 
with an illumination having a spatial structure of a grating. The illumination is 
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Shifted with time in order to simulate temporal motion of the grating. A specially 
designed diffractive optical element is attached to the illumination source so that a 
grating pattern appears in the object plane. The grating pattern is moved by phase 
modulating the light source, since linear phase modulations appear as shifts in a 



far field. 



Additional Considerations 

In reading the above description, persons skilled in the art will 
realize that there are many apparent variations that can be applied to the methods 
and systems described. For example, the integral in Equation 3 may be replaced 
by a discrete Gabor transform over u with window function g; namely, 

yln] = Z^Cp^XC'" -P)y^' (47) 

Similarly, one may use instead a wavelet or Mellin transform, as is well known to 
persons skilled in the art. 

Additionally, the shifts between the multiple images that are 
acquired by the sensing device are not required to be equal. Furthermore, the 
shifts between the multiple images are not required to be horizontally or vertically 
disposed, nor are they required to be disposed in a fixed direction. Wavelet 
transforms and Mellin transforms are particularly advantageous when dealing with 
unequal shifts between acquired images. 

Additionally, the reconstruction algorithm used in the present 
invention, for generating a high resolution image from multiple low resolution 
images, need not be separable. That is, it need not be comprised of horizontal and 
vertical reconstructions. Rather, the reconstruction algorithm used in the present 
invention can be a genuine two-dimensional algorithm. 

In the foregoing, specification, the invention has been described 
with reference to specific exemplary embodiments thereof. It will, however, be 
evident that various modifications and changes may be made to the specific 
exemplary embodiments without departing from the broader spirit and scope of 
the invention as set forth in the appended claims. Accordingly, the specification 
and drawings are to be regarded in an illustrative rather than a restrictive sense. 
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CLAIMS 

What is claimed is: 



^' ^ method for enhancing the resolution of an image sensing 

device, comprising the steps of: 

attaching a mask to. a panel of detectors in an image sensing 

device; 

generating multiple fields of view, the multiple fields of view 
being related to one another by sub-pixel shifts; 

acquiring multiple images with the image sensing device from the 
multiple fields of view; and 

combining the multiple images into an enhanced image of higher 
pixel resolution than the pixel resolutions of the multiple images. 

The method of claim 1 wherein the image sensing device is a 

The method of claun 2 wherein the image sensing device is a 
barcode reader. 

^- method of claim 1 wherein the image sensing device is a 

CCD camera. 

^- The method of claim 1 wherein the mask is a fine transmission 

grating. 

^' "^le method of claim 5 wherein the panel of detectors comprises a 

periodic array of detectors, and the mask is periodic with period equal to that of 
the periodic array of detectors. 

^- The method of claim 5 wherein the panel of detectors comprises a 

periodic array of detectors, and the mask is periodic with period twice that of the 
periodic array of detectors. 

25 



2. 

scanner. 



BNSDOCID: <WO 005920eA1_l_> 



wo 00/59206 PCT/ILOO/00201 

^' The method of claim 5 further comprising the step of detemiining 

the number of slits, the positions of the slits and the widths of the slits in the fine 
transmission grating. 

^- The method of claim 8 wherein said determining step deteraiines 

the number of slits, the positions of the slits and the widths of the slits, so that the 
mask eliminates zeroes in a Fourier transform of a function representing a 
sensitivity of a pixel, within a prescribed range of frequencies. 

10- The method of clainpt 9 wherein the widths of the slits are 
constrained so as not to block more than a prescribed fraction of light energy from 
reaching the detectors. 

11- The method of claim 9 wherein the widths of the slits arc 
constrained so as not to be smaller than a prescribed minimum width. 

The method of claim 9 wherein said determining step uses a 
weight function specifying relative weights to be placed on values for the widths 
of the slits. 

13. The method of claim 9 wherein the mask is of the form 

m(x)^l-f^rectA(8>S(x-xJ . 

14* The method of claim 1 wherein said generating step uses natural 

vibrations of a platform supporting the sensing device. 

15. The method of claim 1 wherein said generating step uses at least 
one mirror to shift fields of view of the image sensing device by sub-pixel shifts. 

16. The method of claim 1 wherein said combining step is applied 
separately to rows and columns of pixel data. 
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The method of claim 16 wherein said combining step uses the 



— ) 
Ax 



to determine the rows and columns of pixel data of the enhanced 



miage. 



18. The method of claim 17 wherein said combining step comprises 

applying an inverse Fourier transfonn to values of U(ei'"). 



19. 

approximation 
2 



The method of claim 16 wherein said combining step uses the 



Ax Ax ^ 



for positive values of c;, and 



0<a> <n , 



2 



Ax ^ ^ Ax ^ 



-Jt <a)<0 , 

for negative values of m, to determine the rows and columns of pixel data of the 
enhanced image. 



20. The method of claim 19 wherein said combining step comprises 

applying an inverse Fourier transform to values of U(e}'^). 



21. 

approximation 



The method of claim 16 wherein said combining step uses the 



Ax ^ Ax 

c.(-y^)^^c,(-yf)-c.(-/^)^^c,(-y 



for positive values of cy, C> < g>< ;r, and 



Ax 



Uie'") « 



A. ^ —zr Ax 



— Ax 
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for negative values of co, -x < a < 0, and wherein the terms S^(-jn) denote the 
difference 

5HC(-jn) = G(-jn) - G(-}(n- ^) , 

to determine the rows and columns of pixel data of the enhanced image. 

The method of claim 21 wherein said combining step comprises 
applying an inverse Fourier transform to values of U(ei'"). 



The method of claim 1 wherein said combining step uses a Gabor 

transform. 



2^- The method of claim 1 wherein said combining step uses a 

wavelet transform. 



25- The method of claim 1 wherein said combining step uses a Mellin 

transform. 



A system for enhancing the resolution of an image sensing 
device, comprising: 

an image sensing device comprising a panel of detectors; 

a mask attached to said panel of detectors; 

a motion generator generating multiple fields of view, the 
multiple fields of view being related to one another by sub-pixel shifts; 

image acquisition circuitry housed within said image sensing 
device acquiring multiple images from the multiple fields of view; and 

a combiner combining the multiple images into an enhanced 
image of higher pixel resolution than the pixel resolutions of the multiple images. 

The system of claim 26 wherein said image sensing device is a 

scanner. 
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28. The system of claim 27 wherein said unage sensing device is a 

barcode reader. 



-9. The system of claim 26 wherein said image sensing device is a 

CCD camera. 



30- The system of claim 26 wherein said mask is a fine transmission 

grating. 

31. The system of claim 30 wherein said panel of detectors comprises 
a periodic array of detectors, and said mask is periodic with period equal to that of 
said periodic array of detectors. 

32. The system of claim 30 wherein said panel of detectors comprises 
a periodic array of detectors, and said mask is periodic with period twice that of 
said periodic array of detectors. 

33. The system of claim 30 further comprising a grating generator 
determining the number of slits, the positions of the slits and the widths of the slits 
in the fine transmission grating. 

34. The system of claim 33 wherein said grating generator determines 
the number of slits, the positions of the slits and the widths of the slits, so that said 
mask eliminates zeroes in a Fourier transform of a function representing a 
sensitivity of a pixel. 

35. The system of claim 34 wherein the widths of the slits are 
constrained so as not to block more than a prescribed fraction of light energy from 
reaching the detectors. 

36. The system of claim 34 wherein the widths of the slits are 
constrained so as not to be smaller than a prescribed minimum width. 
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37. The system of claim 34 wherein said grating generator uses a 

weight function specifying relative weights to be placed on values for the widths 
of the slits. 

The system of claim 34 wherein said mask is of the form 

The system of claim 26 wherein said motion generator uses 
natural vibrations of a platform supporting the sensing device. 

The system of claim 26 wherein said motion generator uses at 
least one mirror to shift jSelds of view of the image sensing device by sub-pixel 
shifts. 



The system of claim 26 wherein said combiner is applied 
separately to rows and columns of pixel data. 

^2. The system of claim 41 wherein said combiner uses the 

approximation 

0(-j — ) 
Ax 

to determine the rows and columns of pixel data of the enhanced image. 



^3. The system of claim 42 wherein said combiner applies an inverse 

Fourier transform to values of U(ei'"). 

The system of claim 41 wherein said combiner uses the 

approximation 



2 



0 <fl; <;r , 



for positive values of o), and 
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7 



for negative values of co, to deteimine the rows and cohiirslf ' 
enhanced image. 



pixel data of the 



^5. The system of claim 44 wherein said combiner comprises 

applymg an mverse Fourier transform to values of U(e}'"). 

46. The system of claim 41 wherein said combiner uses the 

approximation 



for positive values of a* < a, < and 



[ 



^ ^ Ax 



tor negative values ot^.^<^^o.^ wherein U.e tenm denote Uk 

aifrerence 

s^(^jn) = G(-jn) - G(-j(n. yj; , 

to determine the rows and columns of pixel data of the enhanced image. 

47. The system of claim 46 wherein said combiner comprises 

applymg an mverse Fourier transform to values of U(e}'"). 



48. 

transform. 



The system of claim 26 wherein said combiner 



uses a Gabor 



49. 

transform. 



The system of claim 26 wherein said combiner uses a wavelet 



31 



BNSDCXJID: <WO 0059206A1 I > 



wo 00/59206 PCT/ILOO/00201 

50. The system of claim 26 wherein said combiner uses a Mellin 
transform. 

51. A method for enhancing the resolution of an image sensing 
device, comprising the steps of: 

creating replicas of fields of view using an optical element 
attached to an image sensing device; 

acquiring multiple images with the sensing device from the 
replicas of fields of view; and 

combining the multiple images into an enhanced image of higher 
pixel resolution than the pixel resolutions of the multiple images. 

52. The method of claim 51 wherein the optical element is a 
dififractive optical element. 

53. The method of claim 51 wherein the optical element is a 
reflective optical element. 

54. The method of claim 51 wherein the optical element is a 
combined diffractive and reflective optical element. 

55. The method of claim 51 wherein the optical element produces an 
effect of a grating. 



56- The method of claim 51 wherein the optical element is multi- 

faceted. 



57. The method of claim 51 wherein the replicas of fields of views 
are non-overlapping. 

58. A system for enhancing the resolution of an image sensing 
device, comprising: 

an image sensing device; 
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an optical element attached to said image sensing device, said 
optical element being such as to create replicas of fields of view; 

image acquisition circuitry housed within said image sensing 
device acquiring multiple images from the replicas of fields of view; and 

a combiner combining the multiple images into an enhanced 
image of higher pixel resolution than the pixel resolutions of the multiple images. 

The system of claim 58 wherein said optical element is a 
diffractive optical element. 

The system of claim 58 wherein said optical element is a 
reflective optical element. 

The system of claim 58 wherein said optical element is a 
combined diffractive and reflective optical element. 

The system of claim 58 wherein said optical element produces an 
effect of a grating. 



The system of claim 58 wherein said optical element is multi- 
faceted. 



The system of claim 58 wherein the replicas of fields of view are 
non-overlapping. . 
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